Winning combinations of history-dependent games 
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The Parrondo effect describes the seemingly paradoxical situation in which two losing games 
can, when combined, become winning [Phys. Rev. Lett. 85, 24 (2000)]. Here we generalize this 
analysis to the case where both games are history-dependent, i.e. there is an intrinsic memory in 
the dynamics of each game. New results are presented for the cases of both random and periodic 
switching between the two games. 
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I. INTRODUCTION 

The Parrondo effect is the counter-intuitive situa- 
tion whereby individually losing games somehow 'coop- 
erate' to produce a winning game. In particular, these 
losing games can be combined randomly and yet the effect 
still emerges. The intriguing aspect is that randomness 
in this system is acting in a constructive way. Possible 
applications of this effect have been suggested in several 
fields including biogenesis , molecular transport ||^ , 
random walks [|| and biological systems j^. Even in 
the social sciences, 'winning' models for investment have 
been reported (t). 

Consider a gambling game in which the player has a 
time-dependent capital X{t) where t — 0,1,2,..., and 
whose evolution is determined by tossing biased coins. 
The rules as to which coins to toss, and hence the proba- 
bility of winning, are determined by the history, i.e. the 
game is history-dependent. The game can be divided 
into three regimes: winning, losing and fair (for which 
{X(t)) is respectively an increasing, decreasing or con- 
stant function of t) . Parrondo et al ^ considered combi- 
nations of such a history-dependent game B, as described 
above, and a simple biased coin toss (i.e. game A which 
is hXsioiy- independent and hence has no memory). In 
Parrondo et aVs study, game A is defined by the prob- 
ability p of X{t) increasing, where p = \ — ^- Hence 
game A is a losing game for e > 0. Game B is defined 
by the probabilities of four biased coins: {pi,P2jP3jP4}- 
The particular coin played at a given time step depends 
upon the history of the game as shown in Table |. Par- 
rondo et al showed that two losing games A and B can 
be combined to yield a winning game, if the games are 
alternated either periodically or at random. 

The reason that Parrondo's paradox arises for com- 
bined A-B games is that losing cycles in game B are effec- 
tively broken up by the memory less behavior, or 'noise', 
of game A. The question therefore arises: what happens 
if both games are of type B, and hence have losing cy- 
cles? Can the losing cycle in one game break up the 
losing cycle in the other in order to produce 'winning 
dynamics'? Since the answer is not obvious, and since 
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TABLE I: Type B games consist of four coins. The coin to 
be played at time step t is determined by the results of the 
previous two time steps, as shown. 



the Parrondo effect promises to have a variety of appli- 
cations, it is important to establish whether two history- 
dependent games will indeed produce a Parrondo effect. 
This provides the motivation for the present study. 

In this paper, we generalize the analysis of Ref. |l| to 
the case where both games are history-dependent, i.e. 
there is an intrinsic memory in the dynamics of each 
game. We find specific regimes which do indeed exhibit a 
Parrondo effect. New results are presented for the cases 
of both random and periodic switching between the two 
games. The paper is organized as follows. In Sec. II we 
investigate random combinations of two games of type 
B. In Sec. Ill we consider periodic combinations of such 
games. In Sec. IV we investigate the effect of varying the 
switching probability. Section V provides a summary. 



II. RANDOM COMBINATIONS OF 
HISTORY-DEPENDENT GAMES 



We now extend the analysis of Parrondo et al to the 
case of two history-dependent games of type B. We define 
{pi — e} and {qi — e} as the probability sets defining the 
B games and {ri — e} as the probability set defining the 
combined game. We follow Parrondo et al in only con- 
sidering losing games which result by subtracting a small 
quantity e from each of the probabilities that define a fair 
game. As in Ref. Q, we can define a vector Markovian 
process Y{t) based on the capital X{t) as follows: 
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Y{t) = 



X{t) -X{t-1) 
.X{t- 1) - A(t-2) 



(1) 
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Y{t) can take four values (±1,±1). We label the four 
states of Y(t) as shown in Tabic ||. 
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TABLE II: Labels for the four possible states of the Markovian 
process Y{t), where Y(t) is defined in terms of the capital X{t) 
as prescribed by Eq. (hi). 

For e = both B games must be fair. This is achieved 
by the condition (1— P4)(l— Pa)— = ||l[. This yields 
the first two conditions in Eq. (||). For the combined 
game to be winning, we obtain the final condition listed 
in Eq. (ph: 



(1 -P4)(l -P3) =PlP2 

(1 - '74) (1 - 93) = qiq2 

(1 - r4)(l - ra) < rir2 . 



(2) 



If the two B games are combined randomly, the proba- 
bility set for the combined game is given by: 



n ^ api + {I - a)qi 



(3) 



where a is the probability that the game characterized 
by {pi\ will be chosen. We will typically take a — \. 
The third condition in Eq. now becomes: 

(2 - J34 - 94) (2 - P3 - q-i) < {pi + gi)(p2 + 92) • (4) 

Given that we require the initial games to be fair for 
e = 0, we can use the first two conditions in Eq. (||) to 
substitute for pi and qi. Hence: 



(2 - p4 - 94) (2 - p3 - gs) < 
/(1-P4)(1-P3) , (i-g4)(i-g3) 



P2 



12 



{P2 + 92) ■ (5) 



et al using the parameter space (92 = 93, 94), taking for 
the first B game: 



1 10' 4' 4' 10 



Rearranging Eq. 



91 = 



gives: 
(l-g4)(l-g3) 



92 



(6) 



(7) 



qi is a probability and is thus subject to the restriction 
< qi < 1. Therefore in order to be physically realized, 
the game {qi} must be restricted as follows: 



94 > 1 



92 



93-1 

and hence, in the special case where 93 = 92: 

92 



94 > 1 



92-1 



(8) 



(9) 



From Eq. (^) the condition that defines the regions of 
the parameter space in which two fair games combine to 
yield a winning game is given by: 



> 1 + ^l^q2 if 92 > P2 
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Figure |l| depicts the regions of parameter space defined by 
Eqs. (JlO| ) and (||) for {pi} given by Eq. (||). In particular. 
Fig. [^shows the region in which two fair games combine 
to yield a winning game. This is equivalent to the region 
in which two losing games combine to yield a winning 
game for some value of e > 0. In the appendix we derive 
an expression for the maximum value of e for which this 
remains true, Cmax- 

Figure ^ shows emax, given by Eq. ( |A6|) using the same 
game set {pi} as in Fig. |^. The value of emax is shown 
for all possible games {qi} given {pi} defined by Eq. (||). 
This plot demonstrates the robustness of the Parrondo 
effect in the present case of two history-dependent games. 



A. Special case P2 — Ps, q2 — q3 

In order to reduce the number of free variables so that 
the different regions of the parameter space can be dis- 
played in a three dimensional figure, Parrondo et al ^ 
made the restriction p2 = Pa- Here we are going to reduce 
the number of free variables by appealing to the first two 
conditions in Eq. (^. These conditions give pi and qi 
in terms of {pj}, {qj} (j — 2, 3, 4) such that both games 
are fair when e = 0. 

We choose a particular game {pi} and then plot the 
regions in the parameter space (92,93,94) which enclose 
all games {qi} for which the Parrondo effect is observed. 
Initially we treat the special case introduced by Parrondo 



B. General case P2 / Ps, 92 / qs 

Now we drop Parrondo et al's restriction to p2 = pa 
and treat the general case. From Eq. 

Ul + ^q2 iiq,<l + E^q2 
94 <^ (11) 
[>1+(£|^92 if93> 1 + ^^92. 

Figure ^ depicts the regions of parameter space defined 
by Eq. (|llj) and Eq. (|) for {pi} given by Eq. (|). 
Equation (|11|) defines two regions (labelled "I" and "H" 
in Fig. Equation (J^ excludes almost all of region I 
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FIG. 1: Parameter space for combination of two games in 
the special case p2 = P3,q2 ~ qa with {pi} given by Eq. 
(^). Region I depicts the area forbidden by Eq. (^). Region 
II depicts the area where the two games combine to yield 
a winning game. The white region in between represents the 
area excluded by Eq. (jlo|) , in which two fair games combine to 
yield a losing game. The black dot represents the parameters 
for Parrondo et al 's original game as described in Ref. Q for 
which {qi = i.ga = |,(?3 = |,a4 = |}- 




0.4 0.6 
probability: 

FIG. 2: Plot of the maximum value of e for which the two 
losing games {pi} defined by Eq. and {gi , 172, <?2, 94}, com- 
bine to yield a winning game. The white lines indicate the 
locus of points where tmax = 0. The regions in which tmax < 
correspond to two winning games combining to yield a losing 
game. The region to the left of the dashed line is that ex- 
cluded by Eq. (|), as in Fig. |l} 



in this case. An expression for ema x in the general case 
is derived in the appendix, Eq. (|A7|), given that {qi} is a 
fair game for e = 0. In principle we could plot this over 
the 3D axes of Fig. ||. This would be the generalization 
of Fig. |. 

The original combination of game A and game B due 
to Parrondo et al (see Section I) represents a special case 
of our more general treatment. The game considered in 
Ref. [Q corresponds to combining {pi} as defined by Eq. 
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FIG. 3: Regions of parameter space in which two fair games 
combine to yield a winning game for {pi} given by Eq. 
The planes indicate the boundaries of these regions (them- 
selves marked "I" and "11" ) . The unmarked regions are those 
in which the opposite effect occurs. The surface (and the in- 
set) show the boundary of the region forbidden by Eq. 
The black dot represents Parrondo et al 's original game as de- 
scribed in Ref. [^1 for which {qi = i,g2 — |,g3 — |,g4 ~ |}. 



(|) with {qi = i,52 = 5,93 = ^,94 = 5}- In Figures |i| 
and H the black dot represents Parrondo et al 's original 
game. In both cases it can be seen to lie in the region 
where two losing games combine to yield a winning game. 

Thus, we have derived expressions, Eqs. (|l^) and (pd]), 
for the region of the parameter space in which the Par- 
rondo effect is observed to occur in the case of history- 
dependent games being combined with equal probability 
{a = i). We have also derived expr essio ns for the ro- 
bustness of the effect, Eqs. (A6) and (A7). 



III. PERIODIC COMBINATIONS OF 
HISTORY-DEPENDENT GAMES 



Next we investigate periodic combinations of games. 
Rather than randomly selecting the game to be played 
at each time step, game {pi} is played a times and then 
game {qi} is played b times. This cycle is repeated peri- 
odically. Figure ^ shows the capital after 500 times steps, 
resulting from a combination of two games for a range of 
values of a and b. The capital is greater if the games 
are switched more frequently, as found by Parrondo et al 
for the combination of a simple game A and a history- 
dependent game B. The analysis for the periodic case is 
more complex than for the random case because we can 
no longer appeal to a single game formed from a weighted 
average of two games. 

Let the elements of the vector u^, labelled Ui-j, be the 
probability of the game being in state j at time t = i. The 
evolution of the game from to Ui-^-a+b can be described 
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FIG. 4: Value of capital after 500 games averaged over an 
ensemble of 5 x 10^ runs. Games {pi} (defined by Eq. (^) 
and {qt} — {|,|,|,|} are combined periodically for e = 0. 
Game {pi} is played a times, then game {g,;} is played b times 
and so on. 



as follows: 
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(12) 
(13) 

(14) 



Clearly this is not a homogeneous Markovian process be- 
cause the transition matrix is not time-independent. 

In order to proceed we define a homogeneous Marko- 
vian process described by the transition matrix Tq = 
B''A'^ with time steps t — {a-\- b)i, where i = 0, 1, 2, 3. .... 
Consider a large ensemble of games described by Eq. (|l^ ) 
in the long time limit. Select one of these games at ran- 
dom. The stationary state Sq of the homogeneous game 
defined by Tg gives the probability that the selected game 
will be in each of the four possible states (see Table |l|) at 
times t = (a + b)i, where i = 0,1, 2, 3.... This stationary 
state is given by the solution to the equation Sq = ToSq. 
Now we define a new homogeneous process, Ti, by cycli- 
cally permuting the matrices in Tq once to the right (e.g. 
if To = BAA, then Ti = ABA). The stationary state 
of this process gives the probabilities of finding the game, 
selected from the ensemble, in each of the four possible 
states at times t = {a + b)i + 1, where « = 0, 1, 2, 3... 

In the general case the game formed from the n'th 
cyclic permutation of Tq, T„, gives us the probability 
of finding the selected game in each state at times t — 
(a + b)i + n, where i = 0, 1, 2, 3.... This is illustrated by 
Fig. U We can calculate the overall probability of a win 
at time steps t = (a -I- 6) i -I- n by taking the dot product of 



S. (T,= BAA) 
S, (T,= ABA) 
S, (Tj=AAB) 



FIG. 5: Illustration of the time steps at which the stationary 
states S„ of the transition matrices T„ give the probability of 
finding the combined game in each of the possible states. A*' 
is an integer and we take the long time limit (i.e. TV oo). 



the stationary state of the transition matrix T„, with a 
vector formed from the probabilities of each of the coins 
from the game played at that time step. These vectors 
are p = {pi p2 Ps Pi) and q = (91 92 93 94), where p 
corresponds to A in Eq. ( p^ ) and q corresponds to B in 
Eq. (|lj) . The matrix to the right of the product in T„ 
corresponds to the game that will be played at time step 
t = {a + h)i + n. Therefore, if the matrix to the right is 
A we must take the dot product with p. If it is B, we 
must take the dot product with q. 

An expression for the average probability Pwin of a win 
for the combined game, can thus be found by averaging 
over all possible cyclic permutations of Tq. The gradient, 
giad[{X c{t))], is then given by Eq. (Al), as before. The 
resulting expressions are lengthy. Each set of values of 
a and b yields an expression for grad[(Xc(t))] in terms 
of {pi},{qi}, where i — 1,2,3,4. These expressions are 
too complicated to set out here explicitly. However, we 
can numerically plot the analytic equivalent of Figure IJ: 
this is what we have essentially done in Figure ^ The 
lines show the analytic prediction for the average capital 
after 500 time steps, {Xc{500)), found by multiplying Eq. 
( Al) by 500. Each line corresponds to a slice through the 
surface in Fig. ^ at constant b. The error bars indicate 
one standard deviation on the mean over ten ensemble 
averages of the numerical game. Each ensemble average 
comprises an average over 50,000 individual runs. 

We can see that the numerical and analytic results 
agree to within one standard deviation. This confirms 
that the equations generated by the analysis presented in 
this section are indeed correct. Thus, we have derived ex- 
pressions for the robustness of the Parrondo effect when 
two history-dependent games are combined periodically. 



IV. VARYING THE SWITCHING 
PROBABILITY IN THE RANDOM CASE 



We now examine the dependence of the capital on the 
switching probability in the case that the games are ran- 
domly combined. Figure ^ shows the capital after 500 
iterations plotted against the probability per iteration a 
that the game {pi} will be chosen. The curve is symmet- 
ric and demonstrates that the capital is greatest if the 
games are switched with equal probability. When imple- 
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FIG. 6: Comparison of analytic and numerical results for 
periodic switching. The lines show the analytic predictions at 
constant b, the number of steps of game {qi}, for the capital 
after 500 time steps. The error bars show one standard devi- 
ation on the mean over 10 separate ensemble averages, each 
comprising 50,000 numerical runs. 



FIG. 7: The value of the capital after 500 games averaged 
over an ensemble of 500,000 runs. The same games as in Fig. 
^are combined, but this time randomly. Game {pi} is chosen 
with probability a. The solid line shows the theoretical result 
predicted by Eq. (A.3) while the error bars (which are barely 
visible since they are so small) indicate one standard deviation 
about the mean capital over an ensemble of 10 runs. 



meriting the games it is necessary to assign values to the 
results of the coin tosses at times t = —2,-1 in order 
to seed the game. This arbitrary choice introduces tran- 
sients which can slightly bias the final results. However, 
by allowing the game to first run for 100 iterations, this 
effect can be eliminated. 

Th e cu rve in Fig. |^ represents the capital predicted by 
Eq. ( [A^ ) plotted for all a with the same {pi} and {qi}- 
The error bars show one standard deviation either side of 
the mean capital, averaged over an ensemble of 10 runs. 
The agreement between the theoretical curve and numer- 
ical data is therefore better than one standard deviation. 
For a = or 1 we find (Xc(500)) = 0. These values cor- 
respond to just playing one B game or the other. Since 
both games are fair for e = the average capital is zero. 
The fact that the curve is then positive for all values of 
a means that combining the games with any probability 
< a < 1 leads to a winning combined game. 

Parrondo et al showed that combining two losing 
games could lead to a winning game because switching 
between the games can break the cycles which cause the 
"bad" coins to be over-played ||l| . We might therefore ex- 
pect that switching between the games frequently (either 
by reducing a and b or by choosing a switching probabil- 
ity close to 0.5) results in the largest capital gain since 
this will break up the cycles most effectively. This is 
indeed confirmed by the results in Fig. H and R. 



V. SUMMARY 

We have demonstrated that the apparently paradox- 
ical effect of two losing games combining to produce a 
winning game also applies to combinations of two history- 
dependent games. We derived expressions for the regions 
of the parameter space in which the effect is observed for 



both random and periodic combinations of these games. 
We derived expressions for the gradient of the average 
capital and hence the robustness of the Parrondo effect 
for games combined randomly or periodically. 

Our work has therefore expanded the understanding 
of the Parrondo effect by demonstrating its existence for 
new combinations of history-dependent games. We are 
now faced with the more general question as to what 
property of the constituent games guarantees that the 
Parrondo effect will be observed. In addition if we were 
to combine many games of a more general nature than 
those used to date, how could we predict whether the 
effect would emerge? We hope that the present work will 
stimulate further research on such questions, in addition 
to pursuing applications of the Parrondo effect itself. 
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APPENDIX A: DERIVATION OF Emax FOR 
RANDOM COMBINATIONS OF GAMES 

In this appendix we derive expressions for the maxi- 
mum value of e, Cmax) for which two games which are fair 
for e = combine to yield a winning game. 

Let P„in and Piose be the probabilities, per iteration for 
the combined game, that Xc{t) will increase or decrease 
respectively. Then the gradient of the capital line for the 
combined game grad[(Xc(i))] is given by: 

grad[(X,(t))] = 2Pwin - 1 . (Al) 
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One can derive the following expression for P„in for a 
game {r^}: 



p . — 

J win 



ri(r2 + 1 - r4) 



(l-r4)(2ri + l-r-3) + rir2 
Substituting for {ri} from Eq. (^) yields: 

grad[(X,(i))] = 

2 X ... 

[(1 - a)qi + api][{l - a)q2 + ap2] + ... 

. . . [(1 - a)qi + api] x . . . 

. . . (1 - 2(a - l)qi + (a - 1)93 + 2api - aps) x . 
...[! + (!- a)q2 - (1 - a)q4 + a{p2 - P4)] 



(A2) 



[1 + (a - 1)94 - api] 



1 (A3) 



Now we reintroduce e via the transformations pi pi — e, 
qi qi — £ to obtain: 



grad[(Xe(i))] 



2 X 



[api + (1 - Q!)gi - e] [ap2 + (1 - 0)92 - e] + • • • 
. . . [api + (1 - a)qi - e] x . . . 
[-2api + ap3 - 2(1 - a)qi + (1 - ajq^ + e - 1] x 
• ■ • [aip2 -Pi) + (1 - ct){q2 - qi) + 1] 



[ap4 + (1 - a)qi - e - 1] 



1 (A4) 



Any games {pi},{<Zi} defined as above will be losing 
games for all values of e > 0. Thus in order to find the 
maximum value of e for which two losing games combine 
to yield a winning game, we must find the value of e for 



which grad[(Xc(t))] = 0. We shall consider games com- 
bined with equal probability, therefore a — ^. Setting 
grad[(Xc(i))] equal to zero in Eq. (A4) gives: 



_ -4 + P1P2 + 2p3 + 2pi - P3P4 + P2gl + ■ ■ ■ 

2(4 + pi + p2 - J33 - P4 + ■ • • 

■Piq2 + qiq2 + 2^3 - P4'?3 + 2^4 - - '7394 



• 91+92-93 -94J 



(A5) 



Appealing to the condition (Eq. (0)) that {9^} is a fair 
game for e = and in the special case where 92 = 93 and 
P2 = P3, this becomes: 



_ P2[l + 92(1 +pi -P4) - 94] + ■ ■ ■ 
2(1 - 94) + • ■ • 
92[-3 + P4(2 - 92) + 92 +Pig2 + 94] 

292(3+^1 -pi) 

Similarly in the general case, Cmax is given by: 

_ P2[l - 94 + 92PI + 93(94 - 1)] + . . . 

'■""'^ " 2 - 294 + . . . 

■ ■ ■g2[-3 +P4(2 - 93) + 93 + g4] + ■ ■ ■ 
■ • ■ 292(4 + pi + P2 - P3 - P4) + • • • 
■■■g2Pl+P3g2(2-p4-94) 

■ ■ • 292(92 - 93 - 94) + 293(94 - 1) 



(A6) 



(A7) 
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